Policy-Adaptive Estimator Selection for Off-Policy Evaluation

Off-policy evaluation (OPE) aims to accurately evaluate the performance of counterfactual policies using only offline logged data. Although many estimators have been developed, there is no single estimator that dominates others, because estimators' accuracy can vary greatly depending on a given OPE task such as policy, number actions, and noise level. Thus, data-driven selection problem becoming increasingly important significant impact OPE. However, identifying most accurate data quite challenging ground-truth estimation generally unavailable. This paper thus studies this for first time. In particular, we enable an adaptive task, by appropriately subsampling available constructing pseudo useful underlying task. Comprehensive experiments both synthetic real-world company demonstrate proposed procedure substantially improves compared non-adaptive heuristic. Note complete version with technical appendix arXiv: http://arxiv.org/abs/2211.13904.